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REGISTERS FOR DATA TRANSFERS WITHIN A MULTITHREADED PROCESSOR 

BACKGROUND 

Parallel processing is an efficient form of information 
processing of concurrent events in a computing process. 
Parallel processing demands concurrent execution of many- 
programs, in contrast to sequential processing. In the 
context of parallel processing, parallelism involves doing 
more than one thing at the same time. Unlike a serial 
paradigm where all tasks are performed sequentially at a 
single station or a pipelined machine where tasks are 
performed at specialized stations, with parallel processing, 
many stations are provided, each capable of performing and 
carrying out various tasks and functions simultaneously. A 
number of stations work simultaneously and independently on 
the same or common elements of a computing task. 
Accordingly, parallel processing solves various types of 
computing tasks and certain problems are suitable for 
solution by applying several instruction processing units 
and several data streams . 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a processing system. 

FIG. 2 is a detailed block diagram of the processing 
system of FIG. 1 where one of the embodiments of the 
invention may be advantageously practiced. 
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FIG. 3 is a block diagram of a functional pipeline unit 
of the processing system of FIG. 1. 

FIG. 4 is a block diagram illustrating details of the 
processing system of FIG. 1 where one of the embodiments of 
the invention may be advantageously practiced. 

FIG. 5 is a simplified block diagram of a context 
pipeline process. 

FIG. 6 is a flowchart illustrating the process of a 
context pipeline where one of the embodiments of the 
invention may be advantageously practiced. 

FIG. 7 is a flowchart illustrating the process of 
determining the address of the Next Neighbor registers. 

DESCRIPTION 

Architecture : 

Referring to FIG. 1, a computer processing system 10 
includes a parallel, hardware -based multithreaded network 
processor 12. The hardware -based multithreaded processor 12 
is coupled to a memory system or memory resource 14 . Memory 
system 14 includes dynamic random access memory (DRAM) 14a 
and static random access memory 14b (SRAM) . The processing 
system 10 is especially useful for tasks that can be broken 
into parallel subtasks or functions. Specifically, the 
hardware -based multithreaded processor 12 is useful for 
tasks that are bandwidth oriented rather than latency 
oriented. The hardware-based multithreaded processor 12 has 
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multiple functional microengines or programming engines 16a- 
16h (collectively, programming engines 16) each with 
multiple hardware controlled threads that are simultaneously 
active and independently work on a specific task. 
5 The programming engines 16 each maintain program 

counters in hardware and states associated with the program 
counters. Effectively, corresponding sets of context or 
threads can be simultaneously active on each of the 
programming engines 16 while only one is actually operating 

10 at any one time. 

In this example, eight programming engines 16a-16h are 
illustrated in FIG. 1. Each programming engine 16a- 16h 
processes eight hardware threads or contexts. The eight 
programming engines 16a- 16h operate with shared resources 

15 including memory resource 14 and bus interfaces (not shown) . 
The hardware -based multithreaded processor 12 includes a 
dynamic random access memory (DRAM) controller 18a and a 
static random access memory (SRAM) controller 18b. The DRAM 
memory 14a and DRAM controller 18a are typically used for 

20 processing large volumes of data, e.g., processing of 

network payloads from network packets. The SRAM memory 14b 
and SRAM controller 18b are used in a networking 
implementation for low latency, fast access tasks, e.g., 
accessing look-up tables, memory for the core processor 20, 

25 and the like. 

The eight programming engines 16a- 16h access either the 

3 
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DRAM memory 14a or SRAM memory 14b based on characteristics 
of the data. Thus, low latency, low bandwidth data are 
stored in and fetched from SRAM memory 14b, whereas higher 
bandwidth data for which latency is not as important, are 
5 stored in and fetched from DRAM memory 14a. The programming 
engines 16 can execute memory reference instructions to 
either the DRAM controller 18a or SRAM controller 18b. 

The hardware -based multithreaded processor 12 also 
includes a processor core 2 0 for loading microcode control 
10 for the programming engines 16. In this example, although 

other types of processor cores may be used in embodiments of 
this invention, the processor core 20 is an XScale™ based 
architecture, designed by Intel® Corporation, of Santa 
Clara, CA. 

15 The processor core 2 0 performs general -purpose computer 

type functions such as handling protocols, exceptions, and 
extra support for packet processing where the programming 
engines 16 pass the packets off for more detailed processing 
such as in boundary conditions. 

2 0 The processor core 2 0 executes an operating system (not 

shown) . Through the operating system (OS) , the processor 
core 2 0 can call functions to operate on the programming 
engines 16a-16h. For the core processor 20 implemented as an 
XScale™ architecture, operating systems such as Microsoft® 

25 NT real-time of Microsoft® Corporation, of Seattle, 
Washington, VxWorks® real-time operating system of 
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WindRiver®, of Alameda, California, or a freeware OS 
available over the Internet can be used. 

Advantages of hardware multithreading can be explained 
by SRAM or DRAM memory accesses. As an example, an SRAM 
5 access requested by a context (e.g., Thread_0) , from one of 
the programming engines 16, e.g., programming engine 16a, 
will cause the SRAM controller 18b to initiate an access to 
the SRAM memory 14b. The SRAM controller 18b accesses the 
SRAM memory 14b, fetches the data from the SRAM memory 14b, 
10 and returns data to a requesting programming engine 16. 

During an SRAM access, if one of the programming 
engines 16a- 16h has a single thread that could operate, that 
programming engine would be dormant until data was returned 
from the SRAM memory 14b. 
15 By employing hardware context swapping within each of 

the programming engines 16a- 16h, the hardware context 
swapping enables other contexts with unique program counters 
to execute in that same programming engine. Thus, another 
thread e.g., Thread__l can function while the first thread, 
2 0 Thread_0, is awaiting the read data to return. During 

execution, Thread_JL may access the DRAM memory 14a. While 
Thread_l operates on the DRAM unit, and Thread__0 is 
operating on the SRAM unit, a new thread, e.g., Thread_2 can 
now operate in the programming engine 16. Thread_2 can 
2 5 operate for a certain amount of time until it needs to 

access memory or perform some other long latency operation, 
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such as making an access to a bus interface. Therefore, 
simultaneously, the multi-threaded processor 12 can have a 
bus operation, an SRAM operation, and a DRAM operation all 
being completed or operated upon by one of the programming 
5 engines 16 and have one more threads or contexts available 
to process more work. 

The hardware context swapping also synchronizes the 
completion of tasks. For example, two threads can access the 
shared memory resource, e.g., the SRAM memory 14b. Each one 

10 of the separate functional units, e.g., the SRAM controller 
18b, and the DRAM controller 18a, when they complete a 
requested task from one of the programming engine threads or 
contexts reports back a flag signaling completion of an 
operation. When the programming engines 16a- 16h receive the 

15 flag, the programming engines 16a- 16h can determine which 
thread to turn on. 

One example of an application for the hardware -based 
multithreaded processor 12 is as a network processor. As a 
network processor, the hardware -based multithreaded 

2 0 processor 12 interfaces to network devices such as a Media 
Access Controller (MAC) device, e.g., a 10/l00BaseT Octal 
MAC or a Gigabit Ethernet device compliant with IEEE 802.3. 
In general, as a network processor, the hardware -based 
multithreaded processor 12 can interface to any type of 

25 communication device or interface that receives or sends 
large amount of data. The computer processing system 10 
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functioning in a networking application can receive network 
packets and process those packets in a parallel manner. 
Registers in Programming Engines: 

Referring to FIG. 2, one exemplary programming engine 
5 16a from the programming engines 16, is shown. The 

programming engine 16a includes a control store 30, which in 
one example includes a RAM of 4096 instructions, each of 
which is 4 0 -bits wide. The RAM stores a microprogram that 
the programming engine 16a executes. The microprogram in the 
10 control store 3 0 is loadable by the processor core 2 0 (FIG. 
1) - 

In addition to event signals that are local to an 
executing thread, the programming engine 16a employs 
signaling states that are global. With signaling states, an 

15 executing thread can broadcast a signal state to all 

programming engines 16a- 16h. Any and all threads in the 
programming engines can branch on these signaling states. 

As described above, the programming engine 16a supports 
multi -threaded execution of eight contexts. This allows one 

2 0 thread to start executing just after another thread issues a 
memory reference and must wait until that reference 
completes before doing more work. Multi- threaded execution 
is critical to maintaining efficient hardware execution of 
the programming engine 16a because memory latency is 

25 significant. Multi -threaded execution allows the programming 
engines 16 to hide memory latency by performing useful 
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independent work across several threads . 

The programming engine 16a, to allow for efficient 
context swapping, has its own register set, program counter, 
and context specific local registers. Having a copy per 
5 context eliminates the need to move context specific 

information to and from shared memory and programming engine 
registers for each context swap. Fast context swapping 
allows a context to perform computations while other 
contexts wait for input -output (I/O) , typically, external 
10 memory accesses to complete or for a signal from another 
context or hardware unit . 

General Purpose Registers 

The programming engine 16a executes the eight contexts 
15 by maintaining eight program counters and eight context 

relative sets of registers. A number of different types of 
context relative registers, such as general purpose 
registers (GPRs) 32, inter-programming agent registers (not 
shown) , Static Random Access Memory (SRAM) input transfer 
20 registers 34, Dynamic Random Access Memory (DRAM) input 

transfer registers 36, SRAM output transfer registers 38, 
DRAM output transfer registers 40. 

The GPRs 32 are used for general programming purposes. 
The GPRs 3 2 are read and written exclusively under program 
2 5 control. The GPRs 32, when used as a source in an 

instruction, supply operands to an execution datapath 44. 
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The execution datapath 44 can take one or two operands, 
perform an operation, and optionally write back a result. 
The execution datapath 44 includes a content addressable 
memory (CAM) 45. Each entry of the CAM 45 stores a 32-bit 
5 value, which can be compared against a source operand. All 
entries are compared in parallel and the result of the 
lookup is a 6-bit value. 

When used as a destination in an instruction, the GPRs 
32 are written with the result of the execution datapath 44. 
10 The programming engine 16a also includes I/O transfer 

registers 34, 36, 3 8 and 4 0 which are used for transferring 
data to and from the programming engine 16a and locations 
external to the programming engines 16a, e.g., the DRAM 
memory 14a, the SRAM memory 14b, and the like. 

15 

Transfer Registers 

The programming engine 16a also includes transfer 
registers 34, 36, 38 and 40. Transfer registers 34, 36, 38 
and 4 0 are used for transferring data to and from the 
2 0 programming engine 16a and locations external to the 

programming engine, e.g., DRAMs, SRAMs etc. There are four 
types of transfer registers as illustrated in FIG. 2, 
namely, input transfer registers and output transfer 
registers . 

2 5 The input transfer registers, when used as a source in 

an instruction, supply operands to the execution datapath 
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44, whereas output transfer registers are written with the 
result from the execution datapath 44 when utilized as a 
destination in an instruction. 

Local Control and Status Registers (CSRs) 
Local control and status registers (CSRs) 37 are 
external to the execution datapath 44 and hold specific 
purpose information. They can be read and written by special 
instructions (local_csr_ rd and local_csr_wr) and are 
typically accessed less frequently than datapath registers. 

Next Neighbor Registers 

The programming engine 16a also includes one hundred 
and twenty eight (12 8) Next Neighbor (NN) registers, 
collectively referred to as NN registers 35. Each NN 
Register 35, when used as a source in an instruction, also 
supplies operands to the execution datapath 44. Each NN 
register 3 5 is written either by an external entity, not 
limited to, an adjacent programming engine, or by the same 
programming engine 16a where each NN register 35 resides. 
The specific register is selected by a context -relative 
operation where the register number is encoded in the 
instruction, or as a ring operation, selected via, e.g., 
NN_Put (NN write address) and NN_Get (NN read address) in 
the CSR Registers. 

NN_Put registers are used when the previous neighboring 

10 
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programming engine executes an instruction with NN_Put as a 
destination. The NN register selected by the value in this 
register is written, and the value in NN_Put is then 
incremented (a value of 12 7 wraps back to 0) . The value in 
5 this register is compared to the value in NN__Get register to 
determine when to assert NN_Full and NN__Empty status 
signals . 

NN__Get registers are used when the NN register 35 is 
accessed as a source, which is specified in the source field 

10 of the instruction. The NN register selected by the value 
in this register is read, and the value in NN_Put is then 
decremented (a value of 127 wraps back to 0) . The value in 
this register is compared to the value in the NN_Put 
register to determine when to assert NN__Full and NN_Empty 

15 status signals. 

Specifically, when each NN register 35 is used as a 
destination in an instruction, the instruction result data 
are sent out of the programming engine 16a, typically to 
another, adjacent programming engine. On the other hand, 

20 when the NN register 35 is used as a destination in an 

instruction, the instruction result data are written to the 
selected NN Register 35 in the programming engine 16a. The 
data are not sent out of the programming engine 16a as it 
would be when each NN register 3 5 is used as a destination. 

25 Each NN register 3 5 is used in a context pipelining method, 
as described below. 
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A local memory 42 is also used. The local memory 42 
includes addressable storage located in the programming 
engine 16a. The local memory 42 is read and written 
exclusively under program control. The local memory 42 also 
5 includes variables shared by all the programming engines 16. 
Shared variables are modified in various assigned tasks 
during functional pipeline stages by the programming engines 
16a-16h, which are described next. The shared variables 
include a critical section, defining the read-modif y-write 
10 times. The implementation and use of the critical section 
in the computing processing system 10 is also described 
below. 



Functional Pipelining and Pipeline Stages 
15 Referring to FIG. 3, the programming engine 16a is 

shown in a functional pipeline unit 50. The functional 
pipeline unit 5 0 includes the programming engine 16a and a 
data unit 52 that includes data, operated on by the 
programming engine, e.g., network packets 54. The 
2 0 programming engine 16a is shown having a local register unit 
56. The local register unit 56 stores information from the 
data packets 54 . 

In the functional pipeline unit 50, the contexts 58 of 
the programming engines 16a, namely, Programming EngineO.l 
25 (PE0.1) through Programming EngineO.n (PEO.n), remain with 
the programming engine 16a while different functions are 
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performed on the data packets 54 as time 66 progresses from 
time = 0 to time = t . A programming execution time is 
divided into "m" functional pipeline stages or pipe-stages 
60a-60m. Each pipeline stage of the pipeline stages 60a-60m 
performs different pipeline functions 62a, 64, or 62p on 
data in the pipeline. 

The pipeline stage 60a is, for example, a regular time 
interval within which a particular processing function, 
e.g., the function 62a is applied to one of the data packets 
54 . A processing function 62 can last one or more pipelines 
stages 60. The function 64, for example, lasts two pipeline 
stages, namely pipeline stages 6 0b and 60c. 

A single programming engine such as the programming 
engine 16a can constitute a functional pipeline unit 50. In 
the functional pipeline unit 50, the functions 62a , 64, and 
62p move through the functional pipeline unit 5 0 from one 
programming engine (e.g., programming engine 16a), to 
another programming engine (e.g., programming engine 16b), 
as will be described next. 

Referring to FIG. 4, the data packets 54 are assigned 
to programming engine contexts 58 in order. Thus, if "n" 
threads or contexts 58 execute in the programming engine 
16a, the first context 58, u PE0 . 1" completes processing of 
the data packet 54 before the data packets 54 from the 
"PEO.n" context arrives. With this approach the programming 
engine 16b can begin processing the "n+1" packet. 

13 
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Dividing the execution time of the programming engine 
16a, for example, into functional pipeline stages 60a- 60c 
results in. more than one of the programming engines 16 
executing an equivalent functional pipeline unit 70 in 
parallel. The functional pipeline stage 60a is distributed 
across two programming engines 16a and 16b, with each of the 
programming engines 16a and 16b executing eight contexts 
each. 

In operation, each of the data packets 54 remains with 
one of the contexts 58 for a longer period of time as more 
programming engines 16 are added to the functional pipeline 
units 50 and 70. In this example, the data packet 54 remains 
with a context sixteen data packet arrival times (8 contexts 
x 2 programming engines) because context PE0 . 1 is not 
required to accept another data packet 5 8 until the other 
contexts 58 have received their data packets. 

In this example, function 62a of the functional 
pipeline stage 60a can be passed from the programming engine 
16a to the programming engine 16b. Passing of the function 
62a is accomplished by using Next Neighbor registers, as 
illustrated by dotted lines 80a-80c in FIG. 4. 

The number of functional pipeline stages 60a-60m is 
equal to the number of the programming engines 16a and 16b 
in the functional pipeline units 50 and 70. This ensures 
that a particular pipeline stage executes in only one 
programming engine 16 at any one time. 

14 
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Context Pipelining : 

Each of the programming engine 16 supports multi- 
threaded execution of eight contexts. One reason for this is 
5 to allow one thread to start executing just after another 
thread issues a memory reference and must wait until that 
reference completes before doing more work. This behavior is 
critical to maintaining efficient hardware execution of the 
programming engines 16a- 16f because memory latency is 

10 significant. Stated differently, if only a single thread 
execution was supported, the programming engine would sit 
idle for a significant number of cycles waiting for 
references to complete and thereby reduce overall 
computational throughput. Mult i -threaded execution allows a 

15 programming engine to hide memory latency by performing 
useful independent work across several threads . 

The programming engines 16a- 16h (Fig. 1) each have 
eight available contexts. To allow for efficient context 
swapping, each of the eight contexts in the programming 

2 0 engine has its own register set, program counter, and 

context specific local registers. Having a copy per context 
eliminates the need to move context specific information 
to/ from shared memory and programming engine registers for 
each context swap. Fast context swapping allows a context to 

25 do computation while other contexts wait for I/O, typically 
external memory accesses, to complete or for a signal from 
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another context or hardware unit . 

Referring now to FIG. 5, the context for a specific 
assigned task is maintained on the programming engines 16a- 
16c using CAM 45a-45c. The packets are processed in a 
5 pipelined fashion similar to an assembly line using NN 

registers 3 5a-3 5c to pass data from one programming engine 
to a subsequent, adjacent programming engine. Data are 
passed from one stage 90a to a subsequent stage 90b and then 
from stage 90b to stage 90c of the pipeline, and so forth. 

10 In other words, data are passed to the next stage of the 
pipeline allowing the steps in the processor cycle to 
overlap. In particular, while one instruction is being 
executed, the next instruction can be fetched, which means 
that more than one instruction can be in the "pipe" at any 

15 one time, each at a different stage of being processed. 

For example, data can be passed forward from one 
programming engine 16 to the next programming engine 16 in 
the pipeline using the NN registers 35a-35c, as illustrated 
by example in FIG. 5. This method of implementing pipelined 

2 0 processing has the advantage that the information included 
in CAM 45a~45c for each stage 90a-c is consistently valid 
for all eight contexts of the pipeline stage. The context 
pipeline method may be utilized when minimal data from the 
packet being processed must advance through the context 

25 pipeline. 

Referring to FIG. 6, as described above, context 
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pipelining requires that the data resulting from a pipe 
stage, such as pipe stage P, be sent to the next pipe stage, 
e.g., pipe stage P+l (100). Then, Next Neighbor registers 
can be written from the ALU output of the processing engine 
5 16a in pipe stage P (102) , and the Next Neighbor registers 
can be read as a source operand by the next programming 
engine 16b at the pipe stage P+l (104) . 

Referring to FIG. 7, two processes may be used to 
determine the address of the Next Neighbor registers to be 

10 written in the programming engine 16b. In one process, each 
context of the programming engine 16a may write to the same 
Next Neighbor registers for the same context in programming 
engine 16b (2 00) . In another method, a write pointer 
register in the programming engine 16a and a read pointer 

15 register in the programming engine 16b may be used (300) to 
implement an inter processing engine FIFO (302) . The values 
of write pointer register in the programming engine 16a and 
the read pointer register in the programming engine 16b are 
used to produce a full indication checked by the programming 

2 0 engine 16 before inserting data onto the FIFO (3 04) , and an 
empty indication may be used the programming engine 16b 
before removing data from the FIFO (306) . The FIFO Next 
Neighbor configuration may provide the elasticity between 
contexts in the pipe stages P and P+l. When a context in 

2 5 the pipe stage P+l finds the Next Neighbor FIFO is empty, 

that context can perform a No-op function, allowing the pipe 
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stage to maintain a predetermined execution rate or "beat" 
even if the previous pipe stage may not be supplying an 
input at this same rate. 

Other Embodiments: 

In the examples described above in conjunction with 
FIGS. 1-7, the computer processing system 10 may implement 
programming engines 16 using a variety of network 
processors . 

It is to be understood that while the invention has 
been described in conjunction with the detailed description 
thereof, the foregoing description is intended to illustrate 
and not limit the scope of the invention, which is defined 
by the scope of the appended claims. Other aspects, 
advantages, and modifications are within the scope of the 
following claims. 
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WHAT IS CLAIMED IS: 

1. A processor comprising: 

a plurality of programming engines that includes a 
plurality of registers for transferring data from one of the 
plurality of registers residing in an executing programming 
engine to a subsequent one of the plurality of registers 
residing in an adjacent programming engine. 

2. The processor of claim 1 wherein the plurality of 
registers comprises next neighbor registers. 

3. The processor of claim 1 wherein the plurality of 
registers are configured to assign tasks for packet 
processing to the plurality of programming engines. 

4. The processor of claim 1 wherein the plurality of 
registers are configured to establish programming stages 
corresponding to the plurality of programming engines. 

5. The processor of claim 1 wherein the plurality of 
registers establish a plurality of pipelines between the 
programming stages . 

6. The processor of claim 1 wherein the plurality of 
registers maintain the currently operating programming stage 
of the pipeline and the adjacent programming engine is 

19 
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configured to maintain a subsequent programming stage of the 
plurality of pipelines . 

7 . The processor of claim 1 wherein the plurality of 
5 registers support a functional pipeline by a functional 

pipeline control unit that passes functional data among the 
plurality of programming engines. 

8. The processor of claim 7 further comprising a 
10 synchronization unit across the functional pipeline unit. 

9. The processor of claim 7 wherein the functional 
pipeline control unit includes a plurality of functional 
pipeline stages. 

15 

10. The processor of claim 7 wherein each of the 
plurality of functional pipeline stages perform a different 
system function. 

20 11. The processor of claim 7 wherein the plurality of 

programming engines process a data packet in order. 

12 . The processor of claim 7 wherein the data packet 
are assigned to multiple contexts of the plurality of 
25 programming engines. 
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13. The processor of claim 7 wherein the plurality of 
programming engines execute a data packet processing 
function using the functional pipeline unit of the system. 

5 14 . The processor of claim 7 wherein the plurality of 

programming engines perform inter- thread signaling. 

15. A method of transferring data between programming 
engines, the method comprising: 

10 reading data from a plurality of data registers of the 

programming engines for processing the data in a parallel 
processor of a pipeline unit, which supports execution of 
multiple contexts in each of the programming engines; and 

writing data to the plurality of data registers of the 

15 programming engines, the plurality of data registers 
residing in an executing programming engine. 

16. The method of claim 15 further comprising 
providing next neighbor registers for reading data from the 

20 plurality of data registers and writing data to the 
plurality of data registers. 

17. The method of claim 15 further comprising 
assigning tasks for packet processing to the plurality of 

2 5 programming engines. 
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18. The method of claim 15 further comprising 
establishing programming stages corresponding to the 
plurality of programming engines and to establish a 
plurality of pipelines between the programming stages. 

19. The method of claim 15 further comprising 
maintaining the currently operating programming stage of the 
pipeline and configuring the adjacent programming engine to 
maintain a subsequent programming stage of the plurality of 
pipelines . 

20. The method of claim 15 wherein the reading and the 
writing of data includes supporting a functional pipeline by 
a functional pipeline control unit that passes functional 
data among the plurality of programming engines. 

21. A computer program product stored on a computer 
readable medium, the program comprising instructions for 
causing a parallel processor to: 

read data from a plurality of data registers of the 
programming engines for processing data in a parallel 
processor of a pipeline unit, which supports execution of 
multiple contexts in each of the programming engines; and 

write data to the plurality of data registers of the 
programming engines, the plurality of data registers 
residing in an executing programming engine. 
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22. The computer program product "of claim 21 further 
comprising instructions causing the processor to provide 
next neighbor registers for reading data from the plurality 

5 of data registers and writing data to the plurality of data 
registers . 

23. The computer program product of claim 21 wherein 
the plurality of registers execute data transfers using the 

10 pipeline unit. 

24. The computer program product of claim 21 further 
comprising instructions causing the processor to configure 
the plurality of data registers to assign tasks for packet 

15 processing to the plurality of programming engines. 

25. An article comprising: 

a storage medium having stored thereon 
instructions that when executed by a machine results in the 
2 0 following: transfer of data from one 

of a first plurality of registers in an executing 
programming engine to a subsequent one of a second plurality 
of registers residing in an adjacent programming engine; 

assigning of tasks for packet processing to 
2 5 the plurality of programming engines; 
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establishing programming stages corresponding 
to the plurality of programming engines; and 

establishing a plurality of pipelines between 
the programming stages. 

5 

26. The computer program product of claim 2 5 further 
comprising instructions to transfer data from a next 
neighbor. register residing in a currently executing 
programming engine of the programming engines to a 

10 subsequent next neighbor register residing in programming 
engine adjacent to the currently executing programming 
engine . 

27. The computer program product of claim 25 further 
15 comprising instructions for the plurality of registers to 

maintain the currently operating programming stage of the 
pipeline and for the adjacent programming engine to maintain 
a subsequent programming stage of the plurality of 
pipelines . 

20 

28. A multiprocessing system comprising: 

a plurality of programming engines that includes a 
plurality of next neighbor registers configured to transfer 
data from one of a first plurality of registers residing in 
2 5 an executing programming engine to a subsequent one of a 
second plurality of registers residing in an adjacent 
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programming engine and a synchronization unit to process 
data packets across a functional pipeline unit. 

29. The multiprocessing system of claim 28 further 
comprising shared memory locations utilized by a plurality 
of programming stages of the plurality of programming 
engines, the shared variables including a critical section 
defining the read-modify- write time of the memory locations. 

30. The multiprocessing system of claim 28 wherein 
each of the plurality of programming engines further 
includes a content addressable memory (CAM) . 
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